running
  '/usr/lib/R/bin/R --slave --no-restore --no-save --no-restore --file=00_run_all_code.R'

[1] "2021-08-16 20:11:53 EDT"

checkpoint: Part of the Reproducible R Toolkit from Microsoft
https://mran.microsoft.com/documents/rro/reproducibility/
Scanning for packages used in this project
- Discovered 32 packages
All detected packages already installed
checkpoint process complete
---


Running: code/01_process_russian_account_tweets.R

[1] "2021-08-16 20:12:05 EDT"
Building corpus... 
Converting to Lower Case... 
Removing punctuation... 
Removing stopwords... 
Removing numbers... 
Creating Output... 
[1] "2021-08-16 22:12:45 EDT"


Running: code/02_lindvill_warren_labels.R

[1] "2021-08-16 22:12:45 EDT"
[1] "2021-08-16 22:13:03 EDT"


Running: code/03_analyze_russian_account_tweets_scaling_and_mi.R

[1] "2021-08-16 22:13:03 EDT"

Sum not English: 3675083 
[1] "all"
[1] "2009-05-09"
[1] "2016-11-08"
Computing word co-occurrences..
Compressing text..
[1] "LeftTroll"
[1] "2009-05-09"
[1] "2016-11-08"
[1] "RightTroll"
[1] "2009-05-09"
[1] "2016-11-08"
[1] "HashtagGamer"
[1] "2009-05-09"
[1] "2016-11-08"
[1] "NewsFeed"
[1] "2009-05-09"
[1] "2016-11-08"
[1] "all"
[1] "2016-01-01"
[1] "2016-11-08"
Computing word co-occurrences..
Compressing text..
[1] "LeftTroll"
[1] "2016-01-01"
[1] "2016-11-08"
Computing word co-occurrences..
Compressing text..
[1] "RightTroll"
[1] "2016-01-01"
[1] "2016-11-08"
Computing word co-occurrences..
Compressing text..
[1] "HashtagGamer"
[1] "2016-01-01"
[1] "2016-11-08"
Computing word co-occurrences..
Compressing text..
[1] "NewsFeed"
[1] "2016-01-01"
[1] "2016-11-08"
Computing word co-occurrences..
Compressing text..
[1] "2021-08-16 22:43:52 EDT"
Warning message:
S3 methods ‘eigs.dsCMatrix’, ‘eigs.dsRMatrix’, ‘svds.dsCMatrix’, ‘svds.dsRMatrix’ were declared in NAMESPACE but not found 


Running: code/04_combine_account_category_scores_create_keyword_tables.R

[1] "2021-08-16 22:43:52 EDT"

#### ####
#### Table A5
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{llll}
  \hline
D1.pivots.... & D1.....pivots & D2.pivots.... & D2.....pivots \\ 
  \hline
giselleevns & gerfingerpoken & trumpforpresident & unarmed \\ 
  ihatepokemongobecause & thinker & makeamericagreatagain & fatally \\ 
  danageezus & clinton & perfectsliders & police \\ 
  hashtag & tcot & imvotingbecause & officer \\ 
  worldofhashtags & httpstcojeaacre & trumpk & benandjerrysnewflavor \\ 
  midnight & ccot & trumppence & bleepthepolice \\ 
  eat & joeamerica & trumptrain & cop \\ 
  thingseveryboywantstohear & maga & hillaryforprison & policebrutality \\ 
  chrixmorgan & lnyhbt & votetrump & blacklivesmatter \\ 
  pokemon & petefrt & gopdebatesc & acab \\ 
  playing & tlot & draintheswamp & shot \\ 
  ruinadinnerinonephrase & trumptrain & giselleevns & pauloneal \\ 
  ihateitwhen & pjnet & johnatsrs & btp \\ 
  onewordoffmoviequotes & poll & maga & shooting \\ 
  boothprince & rasmussen & lockherup & trueblacknews \\ 
   \hline
\end{tabular}
\end{table}

#### ####
#### Figure 1 (top keywords, conservative)
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{l}
  \hline
keywords\_df...select\_columns. \\ 
  \hline
trumpforpresident \\ 
  makeamericagreatagain \\ 
  perfectsliders \\ 
  imvotingbecause \\ 
  trumpk \\ 
  trumppence \\ 
  trumptrain \\ 
  hillaryforprison \\ 
  votetrump \\ 
  gopdebatesc \\ 
  draintheswamp \\ 
  giselleevns \\ 
  johnatsrs \\ 
  maga \\ 
  lockherup \\ 
   \hline
\end{tabular}
\end{table}

#### ####
#### Figure 1 (top keywords, liberal)
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{l}
  \hline
keywords\_df...select\_columns. \\ 
  \hline
unarmed \\ 
  fatally \\ 
  police \\ 
  officer \\ 
  benandjerrysnewflavor \\ 
  bleepthepolice \\ 
  cop \\ 
  policebrutality \\ 
  blacklivesmatter \\ 
  acab \\ 
  shot \\ 
  pauloneal \\ 
  btp \\ 
  shooting \\ 
  trueblacknews \\ 
   \hline
\end{tabular}
\end{table}

#### ####
#### Table A6
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{llll}
  \hline
D1.pivots.... & D1.....pivots & D2.pivots.... & D2.....pivots \\ 
  \hline
fatally & indieradioplay & blackskinisnotacrime & rapstationradio \\ 
  unarmed & httpstcoemxjgtvv & chaimgoldberg & feat \\ 
  shooting & tycashh & blackoncampus & torae \\ 
  officer & sinice & red-pilled & hiphop \\ 
  benandjerrysnewflavor & thetrudz & nowadays & barz \\ 
  charges & playing & diminish & -fr-o \\ 
  charged & music & antipolicebrutalityday & scarface \\ 
  police & nowplaying & istartcryingwhen & nowplaying \\ 
  pauloneal & listen & beingblackis & checkitout \\ 
  cop & nineoh & fggot & october \\ 
  shot & rapstationradio & altonsterling & contest \\ 
  fixthepolice & boogsmalone & philandocastile & reks \\ 
  dashcam & recklessdondon & wearhoodiefortrayvon & kass \\ 
  bulldoze & rdeyeplug & oscarhasnocolor & xzibit \\ 
  fatal & ogiiiiy & blackpowerbaby & mixtapemppromo \\ 
   \hline
\end{tabular}
\end{table}

#### ####
#### Table A7
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{llll}
  \hline
D1.pivots.... & D1.....pivots & D2.pivots.... & D2.....pivots \\ 
  \hline
renewus & adjusted & afight & islamkills \\ 
  islamkills & rasmussen & trumpisright & brussels \\ 
  afight & incite & crookedcruz & oscarhasnocolor \\ 
  stopislam & bribe & readily & prayforbrussels \\ 
  cosproject & lester & trumpwillwin & oscars \\ 
  jstines & mcclatchy & rkba & stopislam \\ 
  brussels & statespoll & ctot & refugees \\ 
  pjnet & ppollingnumbers & lnyhbt & honorforthebrave \\ 
  irishjoeharriso & holt & perfectlylaura & oscarssowhite \\ 
  readily & emails & trumparmy & europe \\ 
  cruzcrew & manager & tgdn & oscar \\ 
  molonlabe & aide & irishjoeharriso & nocybercensorship \\ 
  ccot & overcharging & ppsellsbabyparts & terrorists \\ 
  nra & probe & noliberalbias & texit \\ 
  makedclisten & allegations & defundpp & religionofpeace \\ 
   \hline
\end{tabular}
\end{table}

#### ####
#### Figure 1 (bottom keywords, political)
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{l}
  \hline
keywords\_df...select\_columns. \\ 
  \hline
unarmed \\ 
  wholeheartedly \\ 
  benandjerrysnewflavor \\ 
  mariowoods \\ 
  fatally \\ 
  danielharris \\ 
  fixthepolice \\ 
  acab \\ 
  policebrutality \\ 
  officer \\ 
  btp \\ 
  police \\ 
  bleepthepolice \\ 
  fggot \\ 
  cop \\ 
   \hline
\end{tabular}
\end{table}

#### ####
#### Figure 1 (bottom keywords, non-political)
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{l}
  \hline
keywords\_df...select\_columns. \\ 
  \hline
rapstationradio \\ 
  feat \\ 
  nowplaying \\ 
  indieradioplay \\ 
  httpstcoemxjgtvv \\ 
  torae \\ 
  hiphop \\ 
  music \\ 
  scarface \\ 
  nineoh \\ 
  album \\ 
  rap \\ 
  -fr-o \\ 
  remix \\ 
  checkitout \\ 
   \hline
\end{tabular}
\end{table}

#### ####
#### Tables A12-A16
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{ll}
  \hline
X2016.words & X2015.words \\ 
  \hline
trump & fukushima \\ 
  black & love \\ 
  https & ukraine \\ 
  hillary & httpt \\ 
  clinton & chernobyl \\ 
  blacklivesmatter & quote \\ 
  httpst & true \\ 
  gloedup & rap \\ 
  islamkills & nuclear \\ 
  giselleevns & npp \\ 
  white & ukrainian \\ 
  danageezus & life \\ 
  pjnet & imho \\ 
  httpstco & lentaruofficial \\ 
  tcot & httptco \\ 
   \hline
\end{tabular}
\end{table}
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{lll}
  \hline
LeftTroll words & X2016.words & X2015.words \\ 
  \hline
blacklivesmatter & black & news \\ 
  black & https & baltimorepost \\ 
  gloedup & trump & sports \\ 
  police & blicqer & braveconwarrior \\ 
  blacktwitter & gloedup & ebbdfcfdeaaedadadfefbeafdce \\ 
  staywoke & talibkweli & local \\ 
  cops & amp & bbsp \\ 
  blicqer & httpst & independent \\ 
  policebrutality & white & politics \\ 
  white & httpstco & httpt \\ 
  bleepthepolice & nowplaying & httptco \\ 
  blm & blackhistorymonth & blackpeopletwitter \\ 
  trayneshacole & btp & isis \\ 
  talibkweli & beingblackis & chris \\ 
  https & thehill & httptc \\ 
   \hline
\end{tabular}
\end{table}
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{lll}
  \hline
RightTroll words & X2016.words & X2015.words \\ 
  \hline
hillary & trump & news \\ 
  trump & hillary & braveconwarrior \\ 
  tcot & islamkills & sports \\ 
  pjnet & clinton & independent \\ 
  obama & https & local \\ 
  clinton & pjnet & chris \\ 
  news & tcot & money \\ 
  realdonaldtrump & ccot & love \\ 
  ccot & amp & bbsp \\ 
  wakeupamerica & brussels & life \\ 
  refugees & httpst & httptco \\ 
  isis & stopislam & make \\ 
  maga & trumpforpresident & httpt \\ 
  via & vote & selfie \\ 
  hillaryclinton & hillaryforprison & politweecs \\ 
   \hline
\end{tabular}
\end{table}
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{lll}
  \hline
HashtagGamer words & X2016.words & X2015.words \\ 
  \hline
midnight & giselleevns & love \\ 
  giselleevns & danageezus & true \\ 
  danageezus & chrixmorgan & life \\ 
  mustbebanned & boothprince & rap \\ 
  igetdepressedwhen & trump & quote \\ 
  ihatepokemongobecause & worldofhashtags & usa \\ 
  chrixmorgan & amp & heart \\ 
  rejecteddebatetopics & phonline & never \\ 
  istartcryingwhen & bunniboila & quotes \\ 
  toavoidworki & midnight & happiness \\ 
  tofeelbetteri & kattfunny & sometimes \\ 
  myolympicsportwouldbe & andyhashtagger & will \\ 
  betteralternativetodebates & annogalactic & success \\ 
  andyhashtagger & https & things \\ 
  donttellanyonebut & gamiliell & mind \\ 
   \hline
\end{tabular}
\end{table}
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 22:49:21 2021
\begin{table}[ht]
\centering
\begin{tabular}{lll}
  \hline
NewsFeed words & X2016.words & X2015.words \\ 
  \hline
news & world & chicago \\ 
  sports & trump & news \\ 
  politics & zika & breaking \\ 
  local & aleppo & local \\ 
  business & warfareww & showbiz \\ 
  foke & environment & newyork \\ 
  chicago & sanders & foke \\ 
  health & topnews & texas \\ 
  topnews & syria & baseball \\ 
  police & clinton & houston \\ 
  world & tech & orleans \\ 
  texas & cruz & atlanta \\ 
  breaking & brexit & reuters \\ 
  tech & rio & detroit \\ 
  says & mosul & astros \\ 
   \hline
\end{tabular}
\end{table}
[1] "2021-08-16 22:49:21 EDT"


Running: code/05_subset_all_russia_tweetids.R

[1] "2021-08-16 22:49:21 EDT"
[1] "2021-08-16 22:50:38 EDT"


Running: code/06_sample_tweets_for_mturk.R

[1] "2021-08-16 22:50:38 EDT"
[1] "2021-08-16 22:50:54 EDT"


Running: code/07_categorize_russian_tweets.R

[1] "2021-08-16 22:50:54 EDT"
INFO [2021-08-16 22:58:16] 2021-08-16 22:58:16 - epoch 1, expected cost 0.2010
INFO [2021-08-16 22:58:20] 2021-08-16 22:58:20 - epoch 2, expected cost 0.1207
INFO [2021-08-16 22:58:23] 2021-08-16 22:58:23 - epoch 3, expected cost 0.0934
INFO [2021-08-16 22:58:26] 2021-08-16 22:58:26 - epoch 4, expected cost 0.0792
INFO [2021-08-16 22:58:30] 2021-08-16 22:58:30 - epoch 5, expected cost 0.0711
INFO [2021-08-16 22:58:33] 2021-08-16 22:58:33 - epoch 6, expected cost 0.0652
INFO [2021-08-16 22:58:36] 2021-08-16 22:58:36 - epoch 7, expected cost 0.0607
INFO [2021-08-16 22:58:39] 2021-08-16 22:58:39 - epoch 8, expected cost 0.0572
INFO [2021-08-16 22:58:43] 2021-08-16 22:58:43 - epoch 9, expected cost 0.0544
INFO [2021-08-16 22:58:46] 2021-08-16 22:58:46 - epoch 10, expected cost 0.0521
INFO [2021-08-16 22:58:49] 2021-08-16 22:58:49 - epoch 11, expected cost 0.0501
INFO [2021-08-16 22:58:53] 2021-08-16 22:58:53 - epoch 12, expected cost 0.0485
INFO [2021-08-16 22:58:56] 2021-08-16 22:58:56 - epoch 13, expected cost 0.0471
INFO [2021-08-16 22:58:59] 2021-08-16 22:58:59 - epoch 14, expected cost 0.0459
INFO [2021-08-16 22:59:03] 2021-08-16 22:59:03 - epoch 15, expected cost 0.0449
INFO [2021-08-16 22:59:06] 2021-08-16 22:59:06 - epoch 16, expected cost 0.0439
INFO [2021-08-16 22:59:09] 2021-08-16 22:59:09 - epoch 17, expected cost 0.0431
INFO [2021-08-16 22:59:12] 2021-08-16 22:59:12 - epoch 18, expected cost 0.0424
INFO [2021-08-16 22:59:15] 2021-08-16 22:59:15 - epoch 19, expected cost 0.0417
INFO [2021-08-16 22:59:19] 2021-08-16 22:59:19 - epoch 20, expected cost 0.0411
INFO [2021-08-16 22:59:22] 2021-08-16 22:59:22 - epoch 21, expected cost 0.0406
INFO [2021-08-16 22:59:25] 2021-08-16 22:59:25 - epoch 22, expected cost 0.0401
INFO [2021-08-16 22:59:28] 2021-08-16 22:59:28 - epoch 23, expected cost 0.0397
INFO [2021-08-16 22:59:32] 2021-08-16 22:59:32 - epoch 24, expected cost 0.0393
INFO [2021-08-16 22:59:35] 2021-08-16 22:59:35 - epoch 25, expected cost 0.0389
INFO [2021-08-16 22:59:38] 2021-08-16 22:59:38 - epoch 26, expected cost 0.0385
INFO [2021-08-16 22:59:42] 2021-08-16 22:59:42 - epoch 27, expected cost 0.0382
INFO [2021-08-16 22:59:45] 2021-08-16 22:59:45 - epoch 28, expected cost 0.0379
INFO [2021-08-16 22:59:48] 2021-08-16 22:59:48 - epoch 29, expected cost 0.0376
INFO [2021-08-16 22:59:52] 2021-08-16 22:59:52 - epoch 30, expected cost 0.0374
INFO [2021-08-16 22:59:55] 2021-08-16 22:59:55 - epoch 31, expected cost 0.0371
INFO [2021-08-16 22:59:58] 2021-08-16 22:59:58 - epoch 32, expected cost 0.0369
INFO [2021-08-16 23:00:01] 2021-08-16 23:00:01 - epoch 33, expected cost 0.0366
INFO [2021-08-16 23:00:05] 2021-08-16 23:00:05 - epoch 34, expected cost 0.0364
INFO [2021-08-16 23:00:08] 2021-08-16 23:00:08 - epoch 35, expected cost 0.0362
INFO [2021-08-16 23:00:11] 2021-08-16 23:00:11 - epoch 36, expected cost 0.0360
INFO [2021-08-16 23:00:15] 2021-08-16 23:00:15 - epoch 37, expected cost 0.0359
INFO [2021-08-16 23:00:18] 2021-08-16 23:00:18 - epoch 38, expected cost 0.0357
INFO [2021-08-16 23:00:21] 2021-08-16 23:00:21 - epoch 39, expected cost 0.0355
INFO [2021-08-16 23:00:24] 2021-08-16 23:00:24 - epoch 40, expected cost 0.0354
INFO [2021-08-16 23:00:28] 2021-08-16 23:00:28 - epoch 41, expected cost 0.0352
INFO [2021-08-16 23:00:31] 2021-08-16 23:00:31 - epoch 42, expected cost 0.0351
INFO [2021-08-16 23:00:34] 2021-08-16 23:00:34 - epoch 43, expected cost 0.0349
INFO [2021-08-16 23:00:37] 2021-08-16 23:00:37 - epoch 44, expected cost 0.0348
INFO [2021-08-16 23:00:41] 2021-08-16 23:00:41 - epoch 45, expected cost 0.0347
INFO [2021-08-16 23:00:44] 2021-08-16 23:00:44 - epoch 46, expected cost 0.0346
INFO [2021-08-16 23:00:47] 2021-08-16 23:00:47 - epoch 47, expected cost 0.0344
INFO [2021-08-16 23:00:50] 2021-08-16 23:00:50 - epoch 48, expected cost 0.0343
INFO [2021-08-16 23:00:54] 2021-08-16 23:00:54 - epoch 49, expected cost 0.0342
INFO [2021-08-16 23:00:57] 2021-08-16 23:00:57 - epoch 50, expected cost 0.0341
[1] "2021-08-16 23:06:04 EDT"


Running: code/08_categorize_russian_tweets_for_irr.R

[1] "2021-08-16 23:06:04 EDT"

#### ####
#### Table A9

#### ####
#### Table A9
[1] "2021-08-16 23:08:20 EDT"


Running: code/09_plot_tweet_scales_over_time.R

[1] "2021-08-16 23:08:20 EDT"


Running: code/10_plot_tweet_labels_over_time.R

[1] "2021-08-16 23:09:47 EDT"
Joining, by = "word"

#### ####
#### Table 1

% Table created by stargazer v.5.2.2 by Marek Hlavac, Harvard University. E-mail: hlavac at fas.harvard.edu
% Date and time: Mon, Aug 16, 2021 - 11:12:50 PM
\begin{table}[!htbp] \centering 
  \caption{} 
  \label{} 
\begin{tabular}{@{\extracolsep{5pt}}lcccc} 
\\[-1.8ex]\hline 
\hline \\[-1.8ex] 
 & \multicolumn{4}{c}{\textit{Dependent variable:}} \\ 
\cline{2-5} 
\\[-1.8ex] & politics + social\_justice & (politics \textgreater  0.5) \textbar  (social\_justice \textgreater  0.5) & politics + social\_justice & (politics \textgreater  0.5) \textbar  (social\_justice \textgreater  0.5) \\ 
\\[-1.8ex] & (1) & (2) & (3) & (4)\\ 
\hline \\[-1.8ex] 
 I(tweet\_date \textgreater = "2016-10-01") & $-$0.16 & $-$0.19 & 0.04 & 0.02 \\ 
  & ($-$0.24, $-$0.09) & ($-$0.28, $-$0.10) & ($-$0.03, 0.11) & ($-$0.07, 0.11) \\ 
  & p = 0.0001 & p = 0.0001 & p = 0.32 & p = 0.64 \\ 
  & & & & \\ 
 Constant & 0.58 & 0.54 & 0.76 & 0.75 \\ 
  & (0.52, 0.63) & (0.47, 0.61) & (0.73, 0.80) & (0.70, 0.80) \\ 
  & p = 0.00 & p = 0.00 & p = 0.00 & p = 0.00 \\ 
  & & & & \\ 
\hline \\[-1.8ex] 
Observations & 450 & 450 & 450 & 450 \\ 
\hline 
\hline \\[-1.8ex] 
\textit{Note:}  & \multicolumn{4}{r}{$^{*}$p$<$0.05; $^{**}$p$<$0.01; $^{***}$p$<$0.001} \\ 
\end{tabular} 
\end{table} 
[1] "Entertainment"
[1] "Politics"
[1] "Social Justice"

#### ####
#### Kappa (Table A8): [1] 0.4241694

#### ####
#### Alpha (Table A8): [1] 0.424276

#### ####
#### Kappa left vs right troll (Table A8): 
% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Mon Aug 16 23:12:51 2021
\begin{table}[ht]
\centering
\begin{tabular}{rrr}
  \hline
 & kappa\_left & kappa\_right \\ 
  \hline
entertainment & 0.37 & 0.29 \\ 
  other & 0.14 & 0.23 \\ 
  politics & 0.47 & 0.55 \\ 
  social\_justice & 0.42 & 0.32 \\ 
   \hline
\end{tabular}
\end{table}

Attaching package: ‘lubridate’

The following object is masked from ‘package:plyr’:

    here

The following object is masked from ‘package:base’:

    date

[1] "2021-08-16 23:13:26 EDT"
Warning message:
In kripp.alpha(t(dfw[, c("X1", "X2", "X3")]), method = "nominal") :
  NAs introduced by coercion


Running: code/11_cluster_russian_accounts.R

[1] "2021-08-16 23:13:26 EDT"
[1] "2021-08-16 23:14:35 EDT"
[1] "2021-08-16 23:14:35 EDT"
